**** ASAQ Chart Generator ****
Author: Siska Fitrianie, Merijn Bruijnes, Amal Abdulrahman, Willem-Paul Brinkman
Corresponding author: Siska Fitrianie, s.fitrianie@tudelft.nl

Interactive Intelligence Group
Department of Intelligent Systems
P.O. Box 5031,
2600 GA Delft,
The Netherlands

To cite:
Siska Fitrianie, Merijn Bruijnes, Amal Abdulrahman, and Willem-Paul Brinkman. 2025. The Artificial Social Agent Questionnaire (ASAQ) - Development and evaluation of a validated instrument for capturing human interaction experiences with artificial social agents. International Journal of Human-Computer Studies (2025), 103482. https://doi.org/10.1016/j.ijhcs.2025.103482

Description of the dataset
------------------------------------------------------------------------------------------------------------------------------------
The ASAQ Chart Generator is one of the output of the research into Siska Fitrianie, Merijn Bruijnes, Amal Abdulrahman, and Willem-Paul Brinkman. 2025. The Artificial Social Agent Questionnaire (ASAQ) - Development and evaluation of a validated instrument for capturing human interaction experiences with artificial social agents. International Journal of Human-Computer Studies (2025), 103482. https://doi.org/10.1016/j.ijhcs.2025.103482

ASAQ is also supported with two charts for reporting ASA questionnaire results and a quick overview of agent profile. The charts are informative visual tools to display the scores of an artificial social agent (ASA) on the 24 constructs/dimensions on a two-dimensional plane, resulted from multi-year efforts involving more than 100 Intelligent Virtual Agent (IVA) researchers worldwide within the OSF work-group of Artificial Social Agent Evaluation Instrument (https://osf.io/6duf7/). The ASAQ Chart can be used for comparing the ASAQ results of up-to 4 ASAs on the original -3 to 3 scale, while the ASAQ Percentile Chart can be used for contrasting the ASAQ results with an ASAQ Representative Set. This set contains dataset of representative ASAs and their unique participants' ASAQ ratings. In the ASAQ Chart, the total score (rounded up), of all the constructs/dimensions is displayed in the middle of the chart which is called the ASA-score.  On the other hand, the middle of the ASAQ Percentile Chart shows the percentile ranking of results relative to the Representative Set.


Abstract of the article Fitrianie et al. (2025)
------------------------------------------------------------------------------------------------------------------------------------
Validating claims and replicating findings on the impact of artificial social agents (ASA), such as virtual agents, conversational agents, and social robots, requires a standardised measurement instrument that researchers can employ in different settings and for various agents. Such an instrument would allow researchers to evaluate their agents and establish insights beyond their specific study context. Therefore, we present the long and short versions of the ASA questionnaire (ASAQ) for evaluating human-ASA interaction on 19 constructs, such as the agent’s believability, sociability, and coherence. It has been developed by an international workgroup with more than 100 ASA-researchers over multiple years who identified community-relevant constructs and associated questionnaire items and examined the questionnaire’s reliability, validity, and interpretability. The result is a questionnaire that can capture more than 80% of the constructs that studies in the intelligent virtual agent community investigate, with acceptable levels of reliability, content validity, construct validity, and cross-validity. We suggest that ASA-researchers use the ASAQ short version to report their agent’s psychographic information and the ASAQ long version to analyse any constructs in-depth that are specifically relevant to their agent or study. Finally, this paper gives instructions for practical use, such as sample size estimations, and how to interpret and present results.

Requirements
------------------------------------------------------------------------------------------------------------------------------------
1. Internet Browser
2. Pdf reader
3. Excel

Content
------------------------------------------------------------------------------------------------------------------------------------
In the folder "ASAQ Chart generator," we provide code to generate ASAQ charts.

- ASAQ Chart generator/asaqchart.html
  The interface to generate an ASAQ chart for visualising ASAQ ratings of one to four ASAs on the original -3 to 3 ASAQ scale
- ASAQ Chart generator/asaqpercchart.html
  The interface to generate an ASAQ chart for contrasting one to four ASAs with the ASAQ representative set 2024
- ASAQ Chart generator/asaqchart.js
  Javascript libraries for displaying ASAQ charts 
- ASAQ Chart generator/howto.png
  A figure that shows the steps to generate an ASAQ chart
- ASAQ Chart generator/csv_asaq_29agents.xlsx
  Examples input for generating ASAQ charts using data from the ASAQ representative set 2024
  - Sheet "long": the mean scores of 24 ASAQ constructs/dimensions of the long ASAQ version of 29 agents
  - Sheet "short": the mean scores of 24 items of the short ASAQ version of 29 agents


Quick Instruction
------------------------------------------------------------------------------------------------------------------------------------
1. See howto.png for quick instruction to generate an ASAQ Chart or ASAQ Percentile Chart
2. Use asaqchart.html for generating an ASAQ chart to compare up-to 4 ASAs
3. Use asaqpercchart.html for generating an ASAQ percentile chart to contrast the ASAQ results (1-4 ASAs) with an ASAQ Representative Set
4. Fill in the short version of ASA Questionnaire (if available) OR 
5. Input 24 scores that are representing the 24 constructs/dimensions in the ASA Questionnaire (decimal is possible). csv_asaq_29agents.xlsx provides examples of input based on the ASAQ Representative Set 2024. 


Revision:
------------------------------------------------------------------------------------------------------------------------------------
11/10/2023
- Add item references using two or three capital letters (see 
Construct/Dimension Numbering Reference above)

10/10/2025
- Change reference to cite and the authorships
- Add the ASAQ Percentile Chart generator codes
- Revise the ASAQ chart generator codes to be more efficient. It can compare up-to 4 ASAs now.